Regimes and mechanisms of transient amplification in abstract and biological neural networks

Neuronal networks encode information through patterns of activity that define the networks’ function. The neurons’ activity relies on specific connectivity structures, yet the link between structure and function is not fully understood. Here, we tackle this structure-function problem with a new conceptual approach. Instead of manipulating the connectivity directly, we focus on upper triangular matrices, which represent the network dynamics in a given orthonormal basis obtained by the Schur decomposition. This abstraction allows us to independently manipulate the eigenspectrum and feedforward structures of a connectivity matrix. Using this method, we describe a diverse repertoire of non-normal transient amplification, and to complement the analysis of the dynamical regimes, we quantify the geometry of output trajectories through the effective rank of both the eigenvector and the dynamics matrices. Counter-intuitively, we find that shrinking the eigenspectrum’s imaginary distribution leads to highly amplifying regimes in linear and long-lasting dynamics in nonlinear networks. We also find a trade-off between amplification and dimensionality of neuronal dynamics, i.e., trajectories in neuronal state-space. Networks that can amplify a large number of orthogonal initial conditions produce neuronal trajectories that lie in the same subspace of the neuronal state-space. Finally, we examine networks of excitatory and inhibitory neurons. We find that the strength of global inhibition is directly linked with the amplitude of amplification, such that weakening inhibitory weights also decreases amplification, and that the eigenspectrum’s imaginary distribution grows with an increase in the ratio between excitatory-to-inhibitory and excitatory-to-excitatory connectivity strengths. Consequently, the strength of global inhibition reveals itself as a strong signature for amplification and a potential control mechanism to switch dynamical regimes. Our results shed a light on how biological networks, i.e., networks constrained by Dale’s law, may be optimised for specific dynamical regimes.


Supporting information S1 Text
Why upper triangular?
The idea behind the use of an upper triangular matrix arises from the real Schur decomposition. Given a connectivity matrix W, one can find the eigenspectrum using the basis of eigenvectors. However, the non-normality of the matrix is lost under this linear transformation. Since we are especially interested in the dynamical regime of transient amplification we have to go beyond the spectrum, and a better way to access the non-normality is to use its Schur decomposition. Indeed, any square matrix is unitarily equivalent to an upper triangular one, and by definition, the minimum norm of the strictly upper part over all such decompositions is its non-normality index. We follow the same idea, but use instead the real Schur transformation. The advantage is that we still have in our hands a real-valued matrix. The disadvantage is that we now have to deal with 2 × 2 blocks along the diagonal. However, it is important to note that W is still orthogonally equivalent to its real Schur transform. This means that the non-normality quantity we are interested in is still preserved, i.e., the dynamical characteristics of transient amplification between the two matrices are not qualitatively different.

Alternative feedforward structures
In the main manuscript, the feedforward structure was taken to be dense and either random, or compatible with the feedforward structure of a stability-optimised circuit.
To check how much our findings depend on those assumptions, we compare two significantly different feedforward structures (S10 Fig). The first is a sparse feedforward structure with probability of connection equal to 0.1. The second structure is comprised of feedforward chains limited to length 2. The simulations show that the results, as a function of the imaginary diameter, do not change qualitatively. However, in the case of the length 2 chains we see that the network is able to amplify much more. Intuitively, this effect might be explained by the fact that all structures are set to have equal norm. Consequently, the weights of the length 2 chains have very strong weights creating a long, strongly coupled and fully connected feedforward chain. Gradually increasing the length of the chains, produces gradually less amplification (not shown) since the weights 1/2 become weaker. The sparse network also has strong weights for the same reason. However, in this case the feedforward chain is sparse and disconnected, yielding less overall amplification.

Biologically plausible network dynamics with strictly positive rates
For mathematically tractability, we considered a linear input/output function to describe continuous firing-rate fluctuations of the network's neurons. For a more realistic comparison, we used a non-linear input/output function with a sigmoidal form (Eq. 2) with bounds defined by r min and r max . In both cases, we considered the output, r(t), as the deviation from the baseline r 0 , and negative values would indicate rates below the baseline. A simple additive transformation can be used so that the network activity is strictly positive. Considering r(t) = f (x(t)) + r min , we can rewrite Eq. 1 as where h is a constant external input onto the network given by The external input h can be interpreted as the baseline input that the neurons receive from upstream areas, which maintains the network's neurons firing at the baseline rate, r 0 .